Methods and Systems to Reduce Privacy Invasion and Methods and Systems to Thwart Same

ABSTRACT

Robust recognition systems to identify potentially identifying information that is contained within a dataset representing a system target are disclosed, together with related method. In an implementations, the systems and methods receive the dataset, identify features in the dataset that have a characteristic indicative of an expected feature, processes the identified features to yield a candidate feature, wherein the candidate feature is one feature contained within the identified features that has the highest probability of containing potentially identifying information.

TECHNICAL FIELD

This disclosure relates to reducing invasions of privacy and topreventing same.

BACKGROUND

Cameras, video and image sensors and scanners, and the like are rapidlyproliferating, and are increasingly used either in connection withembedded computer vision algorithms or to send footage to be processedby such algorithms. Such technologies facilitate ubiquitous automatedrecognition systems (e.g., facial recognition, gait recognition,Automated License Plate Readers or ALPRs, and the like) and can be usedto surveil and/or profile individuals for governmental purposes (e.g.for law enforcement, intelligence gathering, etc.), for commercialpurposes (e.g. to serve ads, to create profiles, to match existing usersto third party profiles, etc.) or the like and are often being appliedmuch more broadly and, perhaps nefariously, in a more covert manner thanoriginally intended.

For example, in the context of license plates, such systems wereoriginally introduced to validate, on a one-on-one basis, that a vehiclewas properly registered and provided a means to distinguish two similarvehicles from each other. In some implementations, ALPRs were justifiedas means to increase the capability of law enforcement to solve crimes.Today ALPR systems have not only become ubiquitous and are used bygovernment entities to continuously monitor citizens who are neverinvolved with, or even suspected of a crime; to make things worse, themajority of the ALPR systems installed in the United States and othercounties are owned, managed and/or operated by private entities to minethe license plate data to build a dynamic map of where vehicles traveland such entities use that information to micro target consumers. Inaddition, such devices and technologies can further systemic inequities,e.g. by facilitating the monitoring of target specific classes ofpersons (e.g. minorities). Consequently, there appears to be alegitimate desire to counteract these and similar technologies torestore the privacy and liberty of citizens in view of the improper useof such systems.

SUMMARY

Robust recognition systems to identify potentially identifyinginformation that is contained within a dataset representing a systemtarget are disclosed, together with related method. In animplementations, the systems and methods receive the dataset, identifyfeatures in the dataset that have a characteristic indicative of anexpected feature, and process the identified features to yield acandidate feature, wherein the candidate feature is one featurecontained within the identified features that has the highestprobability of containing potentially identifying information.

DESCRIPTION OF DRAWINGS

FIG. 1 depicts an embodiment of a system and method, in the context ofan automated license plate reader, for the prevention of a recognitionsystem from identifying one or more features contained in a dataset thatare associated with potentially identifying information.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

In a broad form, the inventor hereof contemplates (i) systems andmethods that prevent recognition systems from identifying one or morefeatures contained in a dataset that are associated with potentiallyidentifying information (“Recognition System Prevention Tools”), and(ii) systems and methods to thwart, or reduce the efficacy of, suchRecognition System Prevention Tools (“Recognition System PreventionTools Thwarter”).

In some implementations, the dataset referenced above can be populatedby the recognition system and includes information representing a systemtarget. In some implementations, the recognition system is programmed toidentify features in a dataset that are likely to be associated withpotentially identifying information by comparing features contained inthe dataset against features expected by the recognition system,wherein, upon location of features in the dataset that are similar tothe features expected by the recognition system, the recognition systemseeks to ascertain information associated therewith in an effort toobtain the potentially identifying information.

In some implementations, the Recognition System Prevention Toolscomprise a modified system target, or the step of modifying the systemtarget, so that the one or more features associated with the potentiallyidentifying information are populated into the dataset so that theydiffer from the features expected by the recognition system. In someimplementations, the modification may be one or both of physical ordigital. For example, (i) a physical modification may include, amongother things, changing the underlayment of the system target to changethe perceived shape and/or color of the one or more features (e.g., byway of a sticker, decal, painting of the underlayment or the like) and(ii) a digital modification may include, among other things, changingthe perceived image collected into the system using digital technologies(e.g., using infrared technology and the like).

In some implementations, the Recognition System Prevention Tools maycomprise decoy features or include the provisioning of decoy features onor about the system target so that the dataset representing the systemtarget are similar to one or more features expected by the recognitionsystem.

In some implementations, the Recognition System Prevention Tools includeboth of the features identified above while other implementations mayemploy at least one of the features. For example, in the context of amethod, the method may include both of the steps of: (i) modifying thesystem target so that the one or more features associated with thepotentially identifying information are populated into the dataset sothat they differ from the features expected by the system; and (ii)modifying the system target to include decoy features in the datasetrepresenting the system target that are similar to one or more featuresexpected by the system.

As described above, the inventor hereof further contemplates methods andsystem that may act to prevent, or reducing the efficacy of the kinds ofRecognition System Prevention Tools herein described (or RecognitionSystem Prevention Tools Thwarter as defined above or “Thwarter”). Insome implementations, and as described above, the recognition systemreviews information in a dataset representing a system target. In someimplementations, the Thwarter processes that information to (i) identifyfeatures therein that that have a characteristic indicative of anexpected feature (e.g., without limitation, generally equivalent in sizeas, or larger than, features expected by the recognition system, similargeometry, or similar color), and (iii) further process such identifiedfeatures to ascertain any potentially identifying information that maybe contained in any such identified features. In other words, in suchexample, the Thwarter is not only reviewing the dataset for featuresthat are similar to the features expected by the system but it reviewingthe dataset for all features that are generally equivalent in size tothe features expected by the system or greater than same and thenpassing those identified features for further processing. It isrecognized that there will be scenarios when the Thwarter identifiesmultiple features for further processing

In an implementation, the Thwarter may comprise the step of removingfeatures in the set of the identified features that are likely to bedecoy features (and that may contain decoy information.) As an example,the Thwarter may review the features in the set of the identifiedfeatures to locate potentially identifying information contained thereinand/or other characteristics of such identified features and comparingthat information and/or such other characteristics against one, some oreach of the following: (i) expected characteristics of such information(e.g., the selection and arrangement, the typeset, the spacing, theshape, or other defining characteristics) such that the system mayidentify something as a decoy features if the characteristics do notalign with the expected characteristics, and (ii) information that isknown to the system as decoy information (e.g., the system may beprogrammed, or otherwise learn through AI or the like, that certaindecoy information is repeatedly used such that it is likely to be decoyinformation.)

In another implementation, the Thwarter may comprise the step ofidentifying features in the set of identified features that containinformation and processing that information to reveal whether suchinformation might be potentially identifying information. As justdescribed, the Thwarter may review the features in the set of theidentified features to locate potentially identifying informationcontained therein and comparing that information against informationthat is more likely to be potentially identifying information. Forexample, the system may make this determination based on a comparison ofsuch information with information in other features in the set of theidentified features and/or the system may make this determination bycomparing such information against a set of information that is flaggedby the system. For example, the set of information that is flagged bythe system may be a list maintained by the system in which the systemseeks.

The description in the remainder of this detailed description describesthe foregoing methods and systems in the context of methods and systemsto prevent (i) license plate recognition systems from identifying alicense plate (the features) contained an image of the license plate(the system target and the dataset) that are associated with the licenseplate number (potentially identifying information), and (ii) tattoorecognition systems from identifying a tattoo (the features) containedin an image of an individual or portions of an individual (the systemtarget and the dataset) that are associated with one or more persons(potentially identifying information). These two examples are butexamples of the potential and expansive embodiments and are intended tobe merely exemplary in nature such that the incorporation herein are, inno way, intended to limit the invention, its application, or uses. Forexample, an additional embodiment, which will not be further discussedbut is referenced merely to illustrate the expansive nature of the broadconcept is facial recognitions (e.g., where the features can be anynumber of facial features, the system target and the dataset can be atleast a portion of a person's face, and the potentially identifyinginformation can be the identity of one or more persons.

License Plate Embodiment

Using automated license plate recognition systems and methods as anexample, and without limiting the breadth of the disclosure, animplementation of a recognition system may undertake the followingsteps: (i) obtaining a frame or sequence of frames (typically becausemovement is detected) to define a system (or scanned) target anddataset, (ii) identifying features in the system target and data setassociated with a rectangular shape of certain proportions (or itshomeomorphic transformations), sometimes with also some additionalattributes (e.g. must be of a certain color or ranges of colors, mustcontain letters or numbers, etc.), (iii) creating a bounding boxtherearound (often with a likelihood of that portion of the image beinga plate), (iv) upon creation of the bounding box, the system may employa mechanism to obtain information associated with the identifiedfeatures; for example, the system may employ optical characterrecognition (OCR) or additional object recognition techniques on theimage contained in the bounding box to yield the potentially identifyinginformation contained there (i.e., the license plate number).

In an implementation, and as described above, a system and method may beemployed to prevent the ALPR system from correctly recognizing andreading the license plate number described in the foregoing paragraph.

With reference to FIG. 1, system 10 may employ a smokescreen 12 that, byway of example, is a device to make the license plate features lessrecognizable by the ALPR system by changing the features in the systemtarget to be different than what the ALPR system expects. For example,and without limitation, an acrylic adhesive may be applied outside theboundaries of the license plate (without altering the plate in anymanner) to change its appearance to a shape other than a rectangle(e.g., a triangle, a circle, or the like). A preferred, but notrequired, objective of the smokescreen is to reduce the algorithm'sconfidence that that particular section of the image is a plate (i.e.,to change the features in the dataset to be different than the featuresexpected by the system). Such smokescreen can be optimized in its designand application to the vehicle to minimize the success of the objectrecognition software that powers the ALPR system.

With continued reference to FIG. 1, instead of the smokescreen ortogether with the smokescreen, system 10 may include a decoy 14. In animplementation, the decoy may be a device that is designed to mimic thefeatures (decoy features) of the target (in this example, a licenseplate) more than the target object itself (especially when a smokescreenis utilized). In some implementations, the decoy may be a sticker, madeto have the same size of a plate, of similar colors, and placed in anopportune area to maximize visibility and readability. Someimplementations may equip such decoy with additional features that maymake it super-salient for the ALPR algorithm (for instance, by adding ahigh contrast border so this decoy plate “pops” as much as possibleagainst the background color of the car.) In some implementations, thedecoy may include specific decoy information meant to inject specificdata in the captured dataset. For example, in the context of ALPRs, thedecoy information may be the value NULL—as such value may be used bycertain recognition systems to label unreadable plates. In this example,the successful injection of the NULL value in the plate reading databaseassociated with certain images or footage may encourage the recognitionto discard such images or footage, or at least to assign such footage toa set of data that needs to be manually verified by a human, therebydefeating the mass automated collection of data. In someimplementations, the decoy information may be placed on or both sides ofthe real information to create a longer string and thereby injectinformation into the captured dataset that is longer than expected bythe system, or is read in a truncated form to match the expected length,hence reducing the reading accuracy

In some implementations, the decoy information may employcharacteristics that are similar in nature to the characteristics of theinformation expected by the system. As an example, the decoy informationmay have one, some, or all of the following: similar font, similarcharacter spacing, similar character color, similar layout, and thelike.

Implementations of Thwarters in the context of the license plate systemwill now be described. As discussed above, in an implementation, aThwarter processes the information in the dataset to (i) identifyfeatures therein that are generally equivalent in size as, or largerthan, features expected by the recognition system (e.g., the size of thelicense plate), and (iii) further process such identified features toascertain any potentially identifying information that may be containedin any such identified features. Referring then to FIG. 1, Thwarter willidentify smokescreen 12 and decoy feature 14 among other potentialfeature (e.g., perhaps the back window and any other feature that islarger than a license plate.)

To simply repeat the disclosure above, in an implementation, theThwarter may comprise the step of removing features in the set of theidentified features that are likely to be decoy features (and that maycontain decoy information.) As an example, the Thwarter may review thefeatures in the set of the identified features to locate potentiallyidentifying information contained therein and comparing that informationagainst information that is known to the system as decoy information(e.g., the system may be programmed to eliminate features that includeNULL.) In some implementations, the Thwarter may compare thecharacteristics of the information contained in the dataset against thecharacteristics of the information expected by the system. For example,the system may eliminate features that include text that has a fontdifferent than the font used on a proper license plate and/or it mayeliminate features that include characters that are spaced apartdifferently than the spacing on a proper license plate.

In another implementation, the Thwarter may comprise the step ofidentifying features in the set of identified features that containinformation and processing that information to reveal whether suchinformation might be potentially identifying information. As justdescribed, the Thwarter may review the features in the set of theidentified features to locate potentially identifying information (NULLand ASDFJKL) contained therein and comparing that information againstinformation that is more likely to be potentially identifyinginformation. For example, the system may make this determination basedon a comparison of such information with information in other featuresin the set of the identified features and/or the system may make thisdetermination by comparing such information against a set of informationthat is flagged by the system. For example, the set of information thatis flagged by the system may be a list maintained by the system in whichthe system seeks.

Further examples of further processing in this context include: (i)identifying information contained in features that is more likely to beassociated with the real plate based on historical or third partyinformation (for instance, if a plate is registered locally, it is morelikely, or if you had multiple readings of that plate then it is morelikely); or (ii) processing the potentially identified information toreview such information for certain character size, font (e.g. needs tobe all CAPS), color, or text structure that is indicative of a realplate.

As described above, the desired information may be nested in decoyinformation that appears on one or both sides of the license plate. Insome implementations, the thwarter may take in all such identifiedinformation and process it to identify whether any iteration of suchidentified information contains expected information. In someimplementation, this kind of processing can be facilitated by comparingiterations of the identified information by comparing each iterationagainst a database of known values. In some implementations, this can befurther accomplished by parsing the identified information into subsetsof the information (e.g., truncated) and identifying whether suchcombinations exist in a database of known values and iterating throughvarious formatives of such subsets.

Tattoo Embodiment

Examples of applying the systems and methods for preventing arecognition system from identifying one or more features contained in adataset that are associated with potentially identifying informationwill now be described in the context of tattoo identification. Asdiscussed above, tattoos are simply yet another one of many examples inwhich the inventive systems and methods can be employed and it is to beunderstood that the inventive systems and methods described herein canbe used for a number of distinguishing features that can be associatedwith one or more persons.

Tattoos can be used to identify one or more individuals (potentiallyidentifying information) that have a particular tattoo (featuresexpected by the system). Taking the principles described above, one orboth of the smokescreen and decoy may be implemented to prevent arecognition system from identifying the particular tattoo. In animplementation, the decoy may employ materials (sticker, makeup, or thelike) that modify features of the tattoo (such as, for example, thecolor, pattern, or other features of the tattoo) so that when it islocated by a recognition system it injects features into the datasetthat are different than the features expected by the system. In someimplementation, and as additional examples, the decoy may employmaterials or methods that alter the tattoo under different lightorientations or conditions (such as, for example, a hologram orinfrared-sensitive pigments) so that the same person, observed underdifferent point of views, under different lighting conditions, or usingdifferent information collection mechanisms will appear to havedifferent tattoos and markings in the context of an automatedrecognition system.

Similar to the example of a license plate, Thwarters may be applied inthe context of tattoos as well. In some implementations, the Thwartermay remove features in the set of identified features to eliminate thosefeatures associated with common tattoos or tattoos that are known by thesystem to be used as decoys.

Various implementations of the systems and techniques described hereincan be realized in digital electronic and/or optical circuitry,integrated circuitry, specially designed ASICs (application specificintegrated circuits), computer hardware, firmware, software, and/orcombinations thereof. These various implementations can includeimplementation in one or more computer programs that are executableand/or interpretable on a programmable system including at least oneprogrammable processor, which may be special or general purpose, coupledto receive data and instructions from, and to transmit data andinstructions to, a storage system, at least one input device, and atleast one output device.

A software application (i.e., a software resource) may refer to computersoftware that causes a computing device to perform a task. In someexamples, a software application may be referred to as an “application,”an “app,” or a “program.” Example applications include, but are notlimited to, system diagnostic applications, system managementapplications, system maintenance applications, word processingapplications, spreadsheet applications, messaging applications, mediastreaming applications, social networking applications, and gamingapplications.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium” and“computer-readable medium” refer to any computer program product,non-transitory computer readable medium, apparatus and/or device (e.g.,magnetic discs, optical disks, memory, Programmable Logic Devices(PLDs)) used to provide machine instructions and/or data to aprogrammable processor, including a machine-readable medium thatreceives machine instructions as a machine-readable signal. The term“machine-readable signal” refers to any signal used to provide machineinstructions and/or data to a programmable processor.

The processes and logic flows described in this specification can beperformed by one or more programmable processors, also referred to asdata processing hardware, executing one or more computer programs toperform functions by operating on input data and generating output. Theprocesses and logic flows can also be performed by special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit). Processors suitable for theexecution of a computer program include, by way of example, both generaland special purpose microprocessors, and any one or more processors ofany kind of digital computer. Generally, a processor will receiveinstructions and data from a read only memory or a random access memoryor both. The essential elements of a computer are a processor forperforming instructions and one or more memory devices for storinginstructions and data. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto optical disks, or optical disks. However, a computer need nothave such devices. Computer readable media suitable for storing computerprogram instructions and data include all forms of non-volatile memory,media and memory devices, including by way of example semiconductormemory devices, e.g., EPROM, EEPROM, and flash memory devices; magneticdisks, e.g., internal hard disks or removable disks; magneto opticaldisks; and CD ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, one or more aspects of thedisclosure can be implemented on a computer having a display device,e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor,e-ink, projection systems, or touch screen for displaying information tothe user and optionally a keyboard and a pointing device, e.g., a mouseor a trackball, by which the user can provide input to the computer.Other kinds of devices can be used to provide interaction with a user aswell; for example, feedback provided to the user can be any form ofsensory feedback, e.g., visual feedback, auditory feedback, or tactilefeedback; and input from the user can be received in any form, includingacoustic, speech, or tactile input. In addition, a computer can interactwith a user by sending documents to and receiving documents from adevice that is used by the user; for example, by sending web pages to aweb browser on a user's client device in response to requests receivedfrom the web browser.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A method for a recognition system to identifypotentially identifying information that is contained within a datasetrepresenting a system target comprising: receiving the dataset;identifying features in the dataset that have a characteristicindicative of an expected feature; processing the identified features toyield a candidate feature, wherein the candidate feature is one featurecontained among the identified features that has the highest probabilityof containing potentially identifying information.
 2. A method for arecognition system as set forth in claim 1, wherein the characteristicindicative of an expected features is selected from the group consistingone or more of the following: (i) whether the size of the feature is atgenerally the same size or a greater size than an expected features,(ii) whether the geometry of the features is generally consistent withthe expected features, and (iii) whether the color of the features isgenerally consistent with the expected features.
 3. A method for arecognition system as set forth in claim 1, wherein the processing stepcomprises: removing the features within the identified features that arelikely to be decoy features.
 4. A method for a recognition system as setforth in claim 1, wherein the removing step comprises: processing thefeatures within the identified features to identify any informationcontained therein and processing any such information to reveal whethersuch information might be potentially identifying information.
 5. Amethod for a recognition system as set forth in claim 4, wherein theprocessing step includes the step of comparing the identifiedinformation with information in a dataset that is pre-populated by thesystem.
 6. A method for a recognition system as set forth in claim 5,wherein the pre-populated dataset contains information that is flaggedby the system such that the system identifies the feature associatedwith the information as the candidate feature if such information of afeature matches information in the pre-populated dataset.
 7. A methodfor a recognition system as set forth in claim 5, wherein thepre-populated dataset contains information that is known by the systemas decoy information such that the system removes the feature associatedwith the information from the set of identified features if suchinformation of a features matches information in the pre-populateddataset.
 8. A method as set forth in claim 1, wherein the system targetincludes a smokescreen that is included in the identified features.
 9. Amethod as set forth in claim 1, wherein the potentially identifyinginformation is a license plate number and the features expected by therecognition system are associated with the shape of a license platehaving the license plate number.
 10. A method as set forth in claim 1,wherein the potentially identifying information is one or more persons,the features expected by the recognition system are associated with oneor more tattoos.